Hanson-wright Inequality and Sub-gaussian Concentration

In this expository note, we give a modern proof of Hanson-Wright inequality for quadratic forms in sub-gaussian random variables. We deduce a useful concentration inequality for sub-gaussian random vectors. Two examples are given to illustrate these results: a concentration of distances between random vectors and subspaces, and a bound on the norms of products of random and deterministic matrices. 1. Hanson-Wright inequality Hanson-Wright inequality is a general concentration result for quadratic forms in sub-gaussian random variables. A version of this theorem was first proved in [2, 9], however with one weak point mentioned in Remark 1.2. In this article we give a modern proof of Hanson-Wright inequality, which automatically fixes the original weak point. We then deduce a useful concentration inequality for sub-gaussian random vectors, and illustrate it with two applications. Our arguments use standard tools of high-dimensional probability. The reader unfamiliar with them may benefit from consulting the tutorial [8]. Still, we will recall the basic notions where possible. A random variable ξ is called sub-gaussian if its distribution is dominated by that of a normal random variable. One can express this by the growth of the moments E |ξ|p = O(p)p/2 as p→∞. This can be quantitatively captured by the sub-gaussian norm of ξ, which is defined as ‖ξ‖ψ2 = sup p≥1 p−1/2(E |X|); ξ is sub-gaussian whenever this quantity is finite. One can similarly define subexponential random variables, by setting ‖ξ‖ψ1 = supp≥1 p−1(E |X|p)1/p. For an m × n matrix A = (aij), recall that the operator norm of A is ‖A‖ = maxx 6=0 ‖Ax‖2/‖x‖2 and the Hilbert-Schmidt (or Frobenius) norm of A is ‖A‖HS = ( ∑ i,j |ai,j |2)1/2. Throughout the paper, C,C1, c, c1, . . . denote positive absolute constants. Theorem 1.1 (Hanson-Wright inequality). Let X = (X1, . . . , Xn) ∈ Rn be a random vector with independent components Xi which satisfy EXi = 0 and ‖Xi‖ψ2 ≤ K. Let A be an n× n matrix. Then, for every t ≥ 0, P { |XAX − EXAX| > t } ≤ 2 exp [ − cmin ( t2 K‖A‖HS , t K2‖A‖ )] . Date: June 21, 2013. M. R. was partially supported by NSF grant DMS 1161372. R. V. was partially supported by NSF grant DMS 1001829 and 1265782. 1 2 MARK RUDELSON AND ROMAN VERSHYNIN Remark 1.2 (Relation to the original Hanson-Wright inequality). Improving upon an earlier result on Hanson-Wright [2], Wright [9] established a slightly weaker version of Theorem 1.1. Instead of ‖A‖ = ‖(aij)‖, both papers had ‖(|aij |)‖ in the right side. The latter norm can be much larger than the norm of A, and it is often less easy to compute. This weak point went unnoticed in several later applications of Hanson-Wright inequality, however it was clear to experts that it could be fixed. Proof. By replacing X with X/K we can assume without loss of generality that K = 1. Let us first estimate p := P { XAX − EXAX > t } . Let A = (aij) n i,j=1. By independence and zero mean of Xi, we can represent XAX − EXAX = ∑